存储在文档中的信息的数字转换是知识的伟大来源。与文本文本相比，嵌入式文档图形的转换，例如图表和地图，已经探讨了。我们介绍了一种方法和系统，用于将文档图表的端到端转换为机器读取的表格数据格式，可以在数字域中容易地存储和分析。我们的方法提取和分析图表以及其图形元素和支持结构，例如图例，轴，标题和标题。我们的检测系统基于神经网络，完全培训了合成数据，消除了数据收集的限制因素。与以前的方法相反，我们的网络使用边界盒检测图形元素，我们的网络特征辅助域特定的热带预测，可以精确地检测不适合矩形边界推定的饼图，线和散点图。定性和定量结果显示出高的稳健性和精确度，改善了以前的流行基准

This work profoundly analyzes discrete self-supervised speech representations through the eyes of Generative Spoken Language Modeling (GSLM). Following the findings of such an analysis, we propose practical improvements to the discrete unit for the GSLM. First, we start comprehending these units by analyzing them in three axes: interpretation, visualization, and resynthesis. Our analysis finds a high correlation between the speech units to phonemes and phoneme families, while their correlation with speaker or gender is weaker. Additionally, we found redundancies in the extracted units and claim that one reason may be the units' context. Following this analysis, we propose a new, unsupervised metric to measure unit redundancies. Finally, we use this metric to develop new methods that improve the robustness of units clustering and show significant improvement considering zero-resource speech metrics such as ABX. Code and analysis tools are available under the following link.

Prior works on improving speech quality with visual input typically study each type of auditory distortion separately (e.g., separation, inpainting, video-to-speech) and present tailored algorithms. This paper proposes to unify these subjects and study Generalized Speech Enhancement, where the goal is not to reconstruct the exact reference clean signal, but to focus on improving certain aspects of speech. In particular, this paper concerns intelligibility, quality, and video synchronization. We cast the problem as audio-visual speech resynthesis, which is composed of two steps: pseudo audio-visual speech recognition (P-AVSR) and pseudo text-to-speech synthesis (P-TTS). P-AVSR and P-TTS are connected by discrete units derived from a self-supervised speech model. Moreover, we utilize self-supervised audio-visual speech model to initialize P-AVSR. The proposed model is coined ReVISE. ReVISE is the first high-quality model for in-the-wild video-to-speech synthesis and achieves superior performance on all LRS3 audio-visual enhancement tasks with a single model. To demonstrates its applicability in the real world, ReVISE is also evaluated on EasyCom, an audio-visual benchmark collected under challenging acoustic conditions with only 1.6 hours of training data. Similarly, ReVISE greatly suppresses noise and improves quality. Project page: https://wnhsu.github.io/ReVISE.

Dual encoders are now the dominant architecture for dense retrieval. Yet, we have little understanding of how they represent text, and why this leads to good performance. In this work, we shed light on this question via distributions over the vocabulary. We propose to interpret the vector representations produced by dual encoders by projecting them into the model's vocabulary space. We show that the resulting distributions over vocabulary tokens are intuitive and contain rich semantic information. We find that this view can explain some of the failure cases of dense retrievers. For example, the inability of models to handle tail entities can be explained via a tendency of the token distributions to forget some of the tokens of those entities. We leverage this insight and propose a simple way to enrich query and passage representations with lexical information at inference time, and show that this significantly improves performance compared to the original model in out-of-domain settings.

Voice Conversion (VC) is the task of making a spoken utterance by one speaker sound as if uttered by a different speaker, while keeping other aspects like content unchanged. Current VC methods, focus primarily on spectral features like timbre, while ignoring the unique speaking style of people which often impacts prosody. In this study, we introduce a method for converting not only the timbre, but also prosodic information (i.e., rhythm and pitch changes) to those of the target speaker. The proposed approach is based on a pretrained, self-supervised, model for encoding speech to discrete units, which make it simple, effective, and easy to optimise. We consider the many-to-many setting with no paired data. We introduce a suite of quantitative and qualitative evaluation metrics for this setup, and empirically demonstrate the proposed approach is significantly superior to the evaluated baselines. Code and samples can be found under https://pages.cs.huji.ac.il/adiyoss-lab/dissc/ .

The ability to compare the semantic similarity between text corpora is important in a variety of natural language processing applications. However, standard methods for evaluating these metrics have yet to be established. We propose a set of automatic and interpretable measures for assessing the characteristics of corpus-level semantic similarity metrics, allowing sensible comparison of their behavior. We demonstrate the effectiveness of our evaluation measures in capturing fundamental characteristics by evaluating them on a collection of classical and state-of-the-art metrics. Our measures revealed that recently-developed metrics are becoming better in identifying semantic distributional mismatch while classical metrics are more sensitive to perturbations in the surface text levels.

Indonesia holds the second-highest-ranking country for the highest number of malaria cases in Southeast Asia. A different malaria parasite semantic segmentation technique based on a deep learning approach is an alternative to reduce the limitations of traditional methods. However, the main problem of the semantic segmentation technique is raised since large parasites are dominant, and the tiny parasites are suppressed. In addition, the amount and variance of data are important influences in establishing their models. In this study, we conduct two contributions. First, we collect 559 microscopic images containing 691 malaria parasites of thin blood smears. The dataset is named PlasmoID, and most data comes from rural Indonesia. PlasmoID also provides ground truth for parasite detection and segmentation purposes. Second, this study proposes a malaria parasite segmentation and detection scheme by combining Faster RCNN and a semantic segmentation technique. The proposed scheme has been evaluated on the PlasmoID dataset. It has been compared with recent studies of semantic segmentation techniques, namely UNet, ResFCN-18, DeepLabV3, DeepLabV3plus and ResUNet-18. The result shows that our proposed scheme can improve the segmentation and detection of malaria parasite performance compared to original semantic segmentation techniques.

Deep neural networks (DNNs) have greatly impacted numerous fields over the past decade. Yet despite exhibiting superb performance over many problems, their black-box nature still poses a significant challenge with respect to explainability. Indeed, explainable artificial intelligence (XAI) is crucial in several fields, wherein the answer alone -- sans a reasoning of how said answer was derived -- is of little value. This paper uncovers a troubling property of explanation methods for image-based DNNs: by making small visual changes to the input image -- hardly influencing the network's output -- we demonstrate how explanations may be arbitrarily manipulated through the use of evolution strategies. Our novel algorithm, AttaXAI, a model-agnostic, adversarial attack on XAI algorithms, only requires access to the output logits of a classifier and to the explanation map; these weak assumptions render our approach highly useful where real-world models and data are concerned. We compare our method's performance on two benchmark datasets -- CIFAR100 and ImageNet -- using four different pretrained deep-learning models: VGG16-CIFAR100, VGG16-ImageNet, MobileNet-CIFAR100, and Inception-v3-ImageNet. We find that the XAI methods can be manipulated without the use of gradients or other model internals. Our novel algorithm is successfully able to manipulate an image in a manner imperceptible to the human eye, such that the XAI method outputs a specific explanation map. To our knowledge, this is the first such method in a black-box setting, and we believe it has significant value where explainability is desired, required, or legally mandatory.

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

超参数优化是识别给定的机器学习模型的适当的超参数配置的过程。对于较小的数据集，可以进行详尽的搜索；但是，当数据大小和模型复杂性增加时，配置评估的数量成为主要计算瓶颈。解决此类问题的有希望的范式是基于替代物的优化。此范式基础的主要思想考虑了超参数空间与输出（目标）空间之间关系的增量更新模型；该模型的数据是通过评估主学习引擎来获得的，例如基于计算机的模型。通过学习近似超参数目标关系，可以使用替代（机器学习）模型来评分大量的超参数配置，并探索除直接机器学习引擎评估的配置空间的一部分。通常，在优化初始化之前选择替代物，并且在搜索过程中保持不变。我们调查了在优化本身期间代孕物质的动态切换是否是选择最合适的基于计算机的大规模在线推荐的最合适的分解模型的实用相关性的明智概念。我们对包含数亿个实例的数据集进行了基准测试，以针对既定基线，例如随机森林和高斯基于过程的替代物。结果表明，替代转换可以提供良好的性能，同时考虑学习引擎评估较少。